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Summary of Review 


As part of an ongoing series of reports by the National Council on Teacher Quality 
(NCTQ), Learning About Learning: What Every New Teacher Needs to Know makes 
broad claims about teacher education based on a limited analysis of textbooks and 
syllabi. The report argues that teacher education materials, specifically educational 
psychology and methods textbooks, are a waste of funds and do not adequately focus 
on what the report identifies as six essential strategies. These inadequacies, the report 
contends, result in ill-prepared teacher candidates lacking in “research-proven instruc- 
tional strategies” (p. vi). The report offers recommendations for textbook publishers, 
teacher education programs, and state departments of education. However, it is not 
grounded in a comprehensive examination of the literature on teaching methods, and 
it fails to validate the evaluative criteria it employs in selecting programs, textbooks, 
and syllabi. The single source it relies on to justify its “six essential strategies” pro- 
vides limited support for NCTQ’s claims. This primary source concludes, with only one 
exception, that the evidence supporting each of the six strategies is only moderate or 
weak. Limiting the analysis to one source that provides only tepid support renders the 
report of little value for improving teacher preparation, selecting textbooks, or guiding 
educational policy. 
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REVIEW OF LEARNING ABOUT LEARNING: 
WHAT EVERY NEW TEACHER NEEDS TO KNOW 


P.L. Thomas, Furman University 
Christian Z. Goering, University of Arkansas 


I. Introduction 


As part of an ongoing series of reports by the National Council on Teacher Quality (NCTQ), 
Learning About Learning: What Every New Teacher Needs to Know! makes broad claims 
about teacher education based on a limited analysis of textbooks and syllabi. The past de- 
cade of high-stakes accountability has brought a new focus on and heightened media and 
political attention to teacher quality. 


The report bases its compilation of teacher education strategies that “work” on this sin- 
gle document from the “What Works Clearinghouse.” The six strategies adapted by 
NCTQ are: (1) pairing graphics with words, (2) linking abstract concepts with con- 
crete representations, (3) posing probing questions, (4) repeating alternating prob- 
lems with their solutions provided and problems that students must solve, (5) distrib- 
uting practice, and (6) assessing to boost retention. The NCTQ report says, “There is 
little debate among scholars about the effectiveness of these six strategies,” yet only 
one of these has “strong” evidence as evaluated by IES, their own source (see Table 1).? 


IES Levels of 
NCTQ Six Strategies that Work ‘ 

Q 5 Effectiveness 
pairing graphics with words moderate 
linking abstract concepts with concrete representations moderate 
posing probing questions strong 
repeating alternating problems with their solutions 

: moderate 
provided and problems that students must solve 
distributing practice moderate 
assessing to boost retention minimal 


Table 1: NCTQ Six Strategies that Work and IES Levels of Effectiveness 
Source: Organizing instruction and study to improve student learning? 
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In fact, IES lists seven categories, ranging from strong to moderate to minimal in effective- 
ness, and the topics listed in the source vary somewhat from the NCTQ report’s strategies. 
Further, the summary of Organizing Instruction and Study to Improve Student Learning 
specifically states, “This guide includes a set of concrete actions relating to the use of in- 
structional and study time that are applicable to subjects that demand a great deal of content 
learning, including social studies, science, and mathematics.” The discrepancies between 
the NCTQ and IES lists are not explained, and the NCTQ report never justifies their claim 
that the strategies accurately represent a consensus among scholars on teaching methods for 
all grade levels and content areas. 


Specifically, the NCTQ report is based on an analysis of 48 textbooks obtained from the 
course syllabi of 48 elementary and secondary teacher preparation programs at 28 institu- 
tions “randomly selected from approximately 490 institutions for which NCTQ had obtained 
full sets of syllabi for professional coursework and student teaching.”4 They evaluate the 
texts in terms of how well they address the six strategies and also base conclusions on analy- 
ses of the syllabi gathered. Conclusions and recommendations are drawn in order to inform 
textbook publishers, teacher education programs, and state departments of education. 


Il. Findings and Conclusions of the Report 


“Looking for the six strategies in these textbooks is akin to looking for six needles in a hay- 
stack,” the report concludes. The primary conclusion of the report is that educational psy- 
chology and methods textbooks in teacher education programs are lacking adequate focus 
on the six strategies identified, and by implication, failing to provide teacher candidates 
with scientifically-based methods: (1) 13% address pairing graphics with words, (2) 25% ad- 
dress linking abstract concepts with concrete representations, (3) 41% address posing prob- 
ing questions, (4) 0% address repeating alternating problems with their solutions provided 
and problems that students must solve, (5) 22% address distributing practice, and (6) 0% 
address assessing to boost retention. Unacknowledged in the NCTQ study is that the narrow 
limits of the IES report significantly skew the analysis of textbooks and syllabi. 


Further, the report acknowledges that textbooks do cover organization of teaching and 
learning, student engagement, cooperative learning, homework, and the importance of prior 
knowledge; however, “many of the topics featured prominently have little research support 
or have been found to have little effect.”5 The lack of “research-based strategies”® in text- 
books, the report suggests, is again linked to the weak attention given to the six strategies 
adapted from Organizing Instruction and Study to Improve Student Learning. 


Included in the report is a related analysis of coursework for teacher candidates, based on 
NCTQ’s examination of course syllabi. From that analysis, the report argues textbooks are 
central to coursework, and thus, that coursework is also inadequate by not addressing the 
six strategies that work. This analysis of syllabi concludes: “Simply reading about the strate- 
gies and discussing them was not sufficient for teachers to understand and use them.”” 
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Late in the report, a case is made for the importance of the six strategies that work before of- 
fering the recommendations and conclusion targeting textbooks authors/publishers, teach- 
er prep programs, and state departments of education. 


Broadly, the report asserts teacher education “tolerates small-scale studies” (echoing the 
federal definition of “scientific” in No Child Left Behind®), and “educators are encouraged 
to practice and preach their own preferred instructional approaches,” the report concludes, 
adding these recommendations?: 


¢ For textbook authors and publishers: “Make textbooks a ‘how to’ guide for 
teachers, presenting what teachers need to know about effectively implementing 
the fundamental instructional strategies in the classroom.””° 


¢ For teacher prep programs: “Teach how to implement the fundamental strat- 
egies and why cognitive science finds them so important and universal.” And, 
“[rJequire teacher candidates to practice instructional strategies to the point of 
mastery.”" 


¢ For state departments of education: “Revise licensing tests to address all six 
fundamental strategies and remove references to practices for which there is no 


research basis.”!? 


Ill. The Report’s Rationale for Its Findings and Conclusions 


Although explicitly expressed as fact (and without justifying their claims), the report’s ratio- 
nale rests on the assertion that teacher education is inadequate, that textbooks are central to 
methods coursework, and that the six strategies provide a credible basis for evaluation of all 
teacher education programs and learning materials. Further, this generalization applies uni- 
formly regardless of level of certification (elementary or secondary) or content being taught. 


lV. The Report’s Use of Research Literature 


The Report’s Appendix G presents a13-page list of references, primarily reflecting the sourc- 
es in the IES study used to anchor NCTQ’s analysis of textbooks and syllabi. The IES study 
itself is useful but limited", as are the sources cited. NCTQ makes no compelling case that 
this one study or the added references are comprehensive or reflective of the current schol- 
arship on “what every teacher needs to know.” Also, the citations are restricted to a very 
narrow view of what counts as research (experimental and quasi-experimental). 


This singular focus on the adapted IES six strategies is not adequately justified. NCTQ’s 
analysis fails to review a wide range and long history of educational research on methods. 
The input and impact of specialized professional associations (SPAs) such as the National 
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Council of Teachers of English, the National Council of Teacher of Math, etc., is notably ab- 
sent. The report ignores a solid consensus on best practice." 


V. Review of Report’s Methods 


The report’s methods are not adequately explained or validated and do not, for example, 
meet the standards for high quality research set forth in their major source document, the 
2007 IES report. Rather, the reader learns on page five that “[t]wo analysts independently 
examined each textbook in the sample to determine if any mention of the six instructional 
strategies was made, even if only in a single sentence.” A great deal of verbiage is spent 
elaborating on how they evaluated the materials but the problem is that this effort is wasted 
if they do not justify their taxonomy. It is unclear how the 48 textbooks represent the whole 
of teacher education course syllabi, much less the overall content of courses contributing to 
the preparation of teachers. While the report notes a random sampling of institutions, the 
initial pool is limited to institutions cooperating with NCTQ, and nowhere is there any ex- 
planation of how representative that initial pool is of teacher education. 


Also troubling, included in the report is one direct refutation by a reviewed textbook’s au- 
thor (Harry Wong) of classifying his textbook as a methods textbook, raising concerns about 
the validity of the textbook analysis. Providing publishers and authors an opportunity to 
interact was the single element of this research where external people were involved. 


VI. Review of Validity of the Findings and Conclusions 


Overall, the NCTQ report fails to justify the validity of its one foundational study and does 
not offer sufficient evidence of its methods for choosing and analyzing textbooks or course 
syllabi in order to reach its conclusions and recommendations. Starting with a different (and 
defensible) set of what teaching strategies work and then analyzing different programs and 
textbooks would likely produce different conclusions and recommendations. Failing its own 
criteria of grounding teacher preparation in research, the NCTQ guide is not an appropriate 
framework for analyzing the complex nature of how people learn or how teachers should 
teach, especially across all grade levels and content areas. From our perspective as English 
teachers and English educators, for example, we would not engage students in repeating 
alternating problems with their solutions provided and problems that students must solve. 


Vil. Usefulness of the Report for Guidance of Policy and Practice 


Teacher education programs, textbooks assigned in educational psychology and methods 
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coursework, and candidate preparedness are all areas worthy of further attention, rigorous 
investigation, and deep analysis. But this report does not substantially contribute to sound 
policy or practice. On the contrary, its narrowness is misleading. 


A valid analysis of teacher education programs as well as the quality of textbooks is certainly 
needed. That undertaking would require a far more comprehensive consideration of best 
practices across all grade levels and content areas. And the analysis of programs and text- 
books would need to meet a much higher bar of validity and generalizability than presented 
in the NCTQ report. 


Ultimately, the report fails to meet the criteria for “scientifically based” it endorses. Despite 
a separate reference section, it does not provide an acceptable or comprehensive research 
base or a thorough explanation of its program evaluation criteria. Significantly, it relies on 
only one, very limited foundational study to the exclusion of a representative and compre- 
hensive study of essential methods—and then makes sweeping key claims without justifi- 
cation. In that sense, the report is akin to taking a limited definition of what a needle is or 
isn’t and then looking for that needle—not in a haystack as the report erroneously states, but 
instead in a hay farmer’s records of where the hay was stacked, to draw conclusions about 
needles and hay. 
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